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Abstract. We define in an axiomatic fashion a Coxeter datum 
for an arbitrary Coxeter group W. Such Coxeter datum will spec- 
ify a pair of reflection representations of W which are not assumed 
to be embeddings of W into the orthogonal group of any real vec- 
tor space. These non-orthogonal representations then give rise to a 
pair of inter-related root systems that generalize the usual orthog- 
onal root systems of Coxeter groups. We obtain comparison results 
between these non-orthogonal root systems and the standard (or- 
thogonal) root systems. Further, we study cones analogous to the 
Tits cones. 



I. Introduction 

For an arbitrary Coxeter system (W,R) in the sense of [13J or [17J, 
it is well known that W can be embedded into the orthogonal group 
of a certain bilinear form ( , ) on a vector space V over the real field R. 
The root system of W in V is a certain VU-stable subset of V whose 
elements correspond to the reflections in W. If the bilinear form ( , ) 
is non-degenerate then (of course) V is IF-isomorphic to its algebraic 
dual Hom(V, R), but since the form is not always non-degenerate it 
is sometimes useful to study both the representation of W on V and 
the contragredient representation of W on Horn(V,R). This motivates 
the approach taken in this paper, in which we consider a pair of real 
vector spaces V\ and Vi linked by a PU-equivariant bilinear pairing sat- 
isfying a few extra conditions (which are guaranteed to hold in the 
case that V\ — V and V2 = Hom(V, R)). We show that in this situa- 
tion W embeds (faithfully) in the general linear groups of each of V\ 
and V2, with both images being generated by involutions. The classi- 
cal theory of (orthogonal) geometric realizations can be recovered from 
this construction as a special case. We define and study generalized 
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root systems arising from such non-orthogonal geometric realizations 
of Coxeter systems, and compare them with the root systems arising 
from the standard geometric realizations. It turns out that it is natural 
to consider root systems in both V% and V 2 simultaneously; these are in 
P^-equivariant bijective correspondence with each other, and are also 
closely related to the classical root system of W in the sense of [2] and 



Let S be an arbitrary set in which each unordered pair {s,t} of 
elements is assigned an m st G Z U {00}, subject to the conditions that 
m ss = 1 (for all s in S), and m st > 2 (for all distinct s,t in S). 
Suppose that V\ and V 2 are vector spaces over the real field R, and 
suppose that there exists a bilinear map ( , ) : V\ x V 2 — >■ R and sets 
III = { a s I s G S } C Vx and U 2 = { (3 S \ s G S } C V 2 such that the 
following conditions hold: 



(C4) (a s , fit) = if and only if (at, (3 S ) = 0, for all s,t G S] 
(C5) X] s g5 ^ sas = ^ w ith -^s > for all s implies X s = for all s, 
and Xlses ^ s @ s = ® w ^h A s > for all s implies X s = for all s. 
Note that (C3) and (C4) together imply that (a s , (3 t ) and (a t , f3 s ) are 
zero if and only if m st = 2. We can also express (C5) more compactly 
as i PLC(ni) and £ PLC(n 2 ), where PLC(A) (the positive linear 
combinations of A) of any set A is defined to be 



{ ^a® I A„ > for all a G A, and \ a > > for some a' G A}. 

Definition 2.1. In the above situation, if conditions (CI) to (C5) are 
satisfied then we call = (S, Vx, V 2 , Tlx, Tl 2 , ( , )) a Coxeter datum. 
The m st (s, t G S) are called the Coxeter parameters of c $ . 

Following [6], we call ^ /ree if both ilx and Il 2 are linearly inde- 
pendent. Throughout this paper, ^ = (S, Vx, V 2 , Hx, H 2 , ( , )) will be 
a fixed Coxeter datum with Coxeter parameters m st , unless otherwise 
stated. We stress that, in general, ^ is not required to be free. 

Remark 2.2. In a Coxeter datum ^ , if we take V 2 = Hom(Vi,]R) 
and take ( , ) to be the natural pairing on V\ x Hom(Vi,R), then the 
conditions (RBI), (RB2) and (RB3) required in the definition of the 
root data of [6] are automatically satisfied in (though, here we have 



HZ]. 



2. Paired reflection representations 



(CI) 
(C2) 
(C3) 



(oc s , P s ) — 1; f° r all s G S; 

(a s , f3 t ) < 0, for all distinct s,t G S; 

for all s, t G S, 
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scaled each element of Hi and U 2 by a factor of 4=). Thus the root 
data in the sense of [B] are special cases of Coxeter datum defined here. 
Further, if we assume that 'io is free then we recover a set of root data 
as given in [12], whereas if we assume that (a s , (3 t ) G Z for all s,t G S 
then we recover a set of root data as given in 



Definition 2.3. Given a Coxeter datum ^ = (S, V\, V 2 , EEi, n 2 , ( , )), 
for each s E S, let pvi(s) and py 2 (s) be the linear transformations on 
V\ and V2 defined by 

p Vl (s)(x) = x - 2(x,f3 s )a s 

for all x EVl, and 

Pv 2 (s)(y) = y- 2(a s ,y)f3 s 

for all y G V 2 . For each i G {1,2} let Rffi) = {pvX s ) \ s ^ S}, and 
let Wii^o) be the subgroup of GL(Vi) generated by 

For each s G 5, it is readily checked that pv 1 (s) and pv 2 ( s ) are 
involutions with pv l (s)(a s ) = —ct s and pv 2 (s)(f3 s ) = —f3 s . Further, we 
have: 

Proposition 2.4. Let x G V\ and y G V 2 . Then for all s G S , 
(p Vl {s){x),p V2 (s){y)) = (x,y). 

□ 

Though c lo lacks freeness in general, nevertheless, conditions (CI), 
(C2) and (C5) of the definition of a Coxeter datum together yield that: 

Lemma 2.5. For each s G S we have a s £ PLC(I1 1 \ and 
0s £ PLC(n 2 \ {/3 S })- I n particular, for distinct s,t G S, the set 
{a s ,a t } is linearly independent, and so is {(3 s ,{3 t }. □ 

For each i G {1,2}, the above lemma yields that pvi( s ) 7^ PVi(t) 
whenever s,t G S are distinct. The following lemma, readily obtained 
from direct calculations, summarizes a few useful results: 

Lemma 2.6. (%) Suppose that s,t G S such that m st ^ {l,oo} ; and 
let 9 = Tr/rrist. If m st 7^ 2 then for each n£N, 



and 



, , sin(2n + 1)6* —cos 9 sm(2n9) 

(Pv 1 (s)p Vl {t)) n (a s ) = — —^as + — — —^a t 

sin 6* \0tt1Ps) sinf 

/ /x f,wnfn\ sin(2n + l)#„ - cos# sin(2n#) ^ 

(p V2 (s)pv 2 (t)) (Ps) = . 2 & + 7 oT ■ fl - A- 

sm0 \ot s ,Pt) sinfe 1 



While if m st = 2 then for each fiGN, 

(p Vl (s)p Vl (t)r(a s ) = (-l) n a s , and (pv 2 (s)p V2 (t)) n (f3 s ) = (-!)"&. 
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(ii) Suppose that s,t E S such that m st = oo. Define 9 = cosh 1 (7), 
where 7 = a/ (a s , /3t)(a t , f3 s ). Then for each neN, 



sinh(2ra+l)6> -7 sinh(2n6>) 



in i,-\n l-fli"/,, \ - J *''.)»« a * + (at,?,) sinhe a *> Z /^7^0> 

1 (2n + l)a s - -^TTjyai, «/# = 0; 



and 



sinh(2n+l)g p , -7 sinh(2nfl) o of -/■ H 
(n ( C \n (+\\n(o\_) sinhfl ^ ^ (a si/ 8 t ) sinhg J / U ' 



□ 



Remarks 2.7. Let ord(pv' i (s)pv i (t)) denote the order of /^(s)/?^^) in 
GL(y i ), for each i G {1, 2}. Then: 

(i) Lemma (i) yields that ord(py (s)py (t)) > m s t when m st ^ oo. 
Indeed, in the subspace with basis {a s ,a t } the following m st elements 

&s, (pvx (s)pvx (t))a 8 , (p Vl (s)p Vl (t)) 2 a S} . . . , (p Vl (s)p Vl {t)) mst ' l a s 

are all distinct, and the same holds in the subspace with basis {/3 S , (3 t }. 

(ii) In a similar way as the above, it follows from Lemma [231 (ii) that 
ord(py(s)py(t)) = 00 when m st = oo. 

The above observations naturally lead to the following: 

Proposition 2.8. Suppose that s,t G S . Then for each i G {1, 2}, 

pv i (s)pv i (t) has order m st in GL(Vj). 

Proof. We give a proof that pv 1 (s)pv 1 (t) has order m st in GL(Vi) be- 
low, and we stress that the same argument will hold for pv 2 ( s )pv 2 (f) 
in GL(V2). Observe that we only need to consider the cases when 
m st {1, oo}, for the statement of the proposition follows readily from 
Remark 12.71 (ii) and the fact that each pv 1 (s), for all s G S, is an 
involution. Next let a G V% be arbitrary. Then 

(pv 1 (s)p Vl (t))(a) = p Vl (s)(a - 2(a,(3 t )a t ) 

= a- 2(a, (3 s )a s - 2(a, (3 t )(a t - 2(a t , f3 s )a s ) 

= a + (4(a, f3 t ) (a t , (3 S ) - 2(a, f3 s ))a s - 2(a, f3 t )a t . 

(2.1) 

If m st = 2, then (12. ip yields that 

(Pvi(s)pvi (*))(<*) = (Pv 1 (*)pv 1 (s))(a) = a-2{a,(3 s )a s -2{a,(3 t }a t , 

so that py^s) and py(t) commute. Hence ord(py 1 (s)py 1 (t)) = 2 when 
m st = 2, and it remains to check the case when m s t > 2. Observe that 
if a = a s and a — a t , then (12.11) yields that 

(PVi{s)pvi(t))M = (4cos 2 (vr/m S i) - l)a s - 2(a s ,f3 t )a t 

and 
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(pviO)pvi(t))Ot) = 2(a t ,f3 s )a s - a t . 

Therefore the action of py 1 (s)py 1 {t) on a t , a} may be represented 

by the following matrix M: 

/ 4cos 2 (vr/m st )-l 2(a t , (3 S ) 4(«, t )(a t , ft) - 2(a, ft) 
M=\ -2(a a ,0t) -1 -2(«,A) 

\ 1 

It is readily checked that M has distinct eigenvalues e m =* , e m =* and 1. 
Hence M has order m st , and so (py (s)pyj (t)) mst = 1 in GL(Vi). Fi- 
nally, in view of Remark 12.71 (i), it follows that ord(py (s)py 1 (t)) is 
precisely m st . □ 

Remark 2.9. Given a Coxeter datum ^ = ( S, V±, V 2 , Hi, H 2 , (, ) ) with 
Coxeter parameters m st (where s,t G S), let (W, R) be a Coxeter sys- 
tem in the sense of [13] or [T7j with i? = {r s |sGS'} being a set of invo- 
lutions generating W subject only to the condition that the order of the 
product r s r t is m st whenever s, t are in S with m st ^ oo. Then Propo- 
sition 12.81 yields that there are group homomorphisms f\ : W — > W\ (^) 
and f 2 : W -> W 2 { c £) satisfying f x {r s ) = pv 1 (s) and f 2 (r s ) = pv 2 {s) for 
all s G S. 

The principal result of this section is the following: 

Theorem 2.10. Let (W,R), f\ and f 2 be as the above. Then f\ and 
f 2 are isomorphisms; that is, (W 1 ( ( t?),R 1 ( ( t?')) and (W 2 (^), R 2 (^)) are 
both Coxeter systems isomorphic to (W,R). 

For each i G {1,2}, since Wii^o) is generated by the elements of 
Riiff), it follows readily that each fi is surjective. Thus only the injec- 
tivity of fi needs to be checked. Before we can do so, a few elementary 
results are needed. First we have a result easily obtained from the 
formulas in Lemma [2.61 

Lemma 2.11. Suppose that s,t G S, and let n be an integer such that 
< n < m st . Write 

• • • Pvi {t)pvi {s)pvi (t) (a s ) = \ n a s + p n a t 



n factors 

and 



Pv 1 {s)pv 1 {t)pv 1 {s){at) = X' n a s + p' n a t . 



n factors 

Then all four constants X n , p n , X' n and p! n are non-negative. □ 

Remark 2.12. The same argument applies equally well if we replace 
in the above lemma, respectively, pvi(s), pvi(t), « s and a t by py 2 (s), 
pv 2 {t), ft and ft. 
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Let W, fi and f 2 be as in Remark 12. 91 Then f\ and f 2 give rise to 
PU-actions on V\ and V 2 in the following way: wx = (fi(w))(x) for all 
w G W and x G Vi, and = (f 2 (w))(y) for all w G W and y G V 2 . 
Let £: W — > N be the length function of with respect to R. For 
u> G W we say that an expression of the form w — r Sl ■ ■ ■ r Sl (where 
si, . . . si G S) reduced if £(w) = I. For any w G W, an easy induction 
on £(w) yields the following extension to Proposition 12.41 

Lemma 2.13. Given a Coxeter datum ^ = (S, V\, V 2 , Tl\, II2, (, ) ), 

and let W be as in in Remark \2.tA Then (x, y) = (wx, wy) for all 
w G W, x G Vi, and y G V 2 . □ 

Using the same argument as in [15J, we may deduce: 

Proposition 2.14. ([151 Theorem, Lecture 1]) Let W be as the above, 
and letweW and s G S. If £{wr s ) > £(w) then wa s G PLC(rii). □ 

Now we are ready to complete the proof of Theorem I2.10t 

Proof of Theorem \2.KA Suppose, for a contradiction, that the kernel 
of f\ is nontrivial, and choose w in the kernel of fi with w 7^ 1. Then 
£(w) > 0, and we may write w = w'r s for some s G S and w' G W 
with £{w') = £{w) — 1. Since £{w'r s ) > £(w'), Proposition 12.141 yields 
w'a s G PLC(rii). But then 

a s = wa s = {w'r s )a s = w'{r s a s ) = w'(—a s ) = —w'a s , 

and hence = a s + w'a s G PLC(Ili), contradicting condition (C5) of 
a Coxeter datum. In an entirely similar way it can be shown that the 
kernel of f 2 is trivial. □ 

Let W and R be as in Remark 12.91 We call W the abstract Coxeter 
group determined by the Coxeter parameters of the Coxeter datum 
and we call (W, R) the abstract Coxeter system associated with 
Observe that Theorem 12.101 yields that the H^-actions on V± and V 2 
induced by the isomorphisms fi and f 2 are faithful. 

3. Root Systems and Canonical Coefficients 

Definition 3.1. Suppose that ^ = ( S, V\, V 2 , Hi, n 2 , ( , ) ) is a Coxeter 
datum, and suppose that (W, R) is the associated abstract Coxeter 
system. 

(i) Define $i(^) = = {wa s | w G W, s G S}, and similarly 
define $ 2 (^) = WU 2 = { w(3 s | w G W, s G S }. For each i G {1,2}, 
we call $i(^) the root system of W in Vi, and its elements the roots of 
W in Vi. We call Hi the set of simple roots in $j(^), and we say that 
Hi forms a root basis for $j(^). 

(ii) Set $+(^) = ^(T) n PLC(ILJ, and $"(^) = -$+(^) for each 
i G {1, 2}. We call $ 4 + (^) the set of positive roots in $i(^), and $~(^) 
the set of negative roots in $ i ( < ^'). 
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Given the above notations, for each % G {1,2}, we adopt the tra- 
ditional diagrammatic description of simple roots 11^: draw a graph 
that has one vertex for each s G S, and join the vertices corresponding 
to s, t G S by an edge labelled by m st if m st > 2. The label m st is 

often omitted if m st = 3. Thus the diagram corresponds to 

III = { a r , a s , a t } and U 2 = { (3 r , (3 S , (3 t }. Suppose that 

(a r , p.) = -1/4, (a s , fa) = -1/6, (a t , &) = -1/10, 

(a s , r ) = -1, (a t , (5 S ) = -3/2, (a r , ft) = -5/2. 

Then 

(r>r s )a r = r r (a r + \a s ) = \a s \ 
(r t r r r s )a r = r t (\a s ) = \a s + \a t \ 
(r s r t r r r s )a r = r s (\a s + \a t ) = ~a t ; 
(r r r s r t r r r s )a r = r r {\a t ) = \a t + ±a r ; 
(r t r r r s r t r r r s )a r = r t (\a t + ±a r ) = ±a r . 

In particular, we notice from the above that it is possible for a non- 
trivial positive scalar multiple of a root to also be a root, lying in the 
same jy-orbit as the root itself. Clearly if wa = Xa where a G $i(^) 
then w n a = X n a for all n G N. Since it is quite possible that A ^ ±1, 
it follows that there could well be infinitely many non-trivial scalar 
multiples of a in $i(^). Further, all roots in the H^-orbit of a will 
possess this same property. Of course, the same situation could arise 
in $2(^) as well. This is one of the features setting $i(^) and $2(X) 
apart from the classical root systems studied in [2], [13] or [T7] . 
Similar to [17] , Proposition 12. 141 yields the following 

Lemma 3.2. Suppose that ^ = ( S, Vi, V 2 , III, n 2 , ( , )) is a Coxeter 
datum, and suppose that (W, R) is the associated abstract Coxeter sys- 
tem, (i) §0) = $+(^) w K(^)> f or each » e {1,2}, where ttl 
denotes disjoint union. 
(ii) If w E W and s G 5, £/ien 

_ f^H + 1 «/to s G $^(^) ; and w/3 s G 

□ 

Remark 3.3. Part (ii) of the above lemma implies that for w G W and 
s e S, wa s G if and only if G $2 C^)* ancl wa s G if 

and only if w/3 s G $ 2 (^). 

Remark 3.4. Since c € is not assumed to be free, although we know 
from Lemma 13.21 (i) that each root in (for each % G {1,2}) 

is expressible as a linear combination of simple roots from ITj with 
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coefficients all being of the same sign, that expression needs not be 
unique. Thus the concept of the coefficient of a simple root in a given 
root is potentially ambiguous. To obtain a canonical way of expressing 
roots in terms of simple roots, we employ a construction similar as 
those of [UJ, [16] and [21]. We define a free Coxeter datum ^" on the 
same set of Coxeter parameters as those of if. Then both if and the 
free Coxeter datum if' correspond to the same abstract Coxeter system 
(W,R). It turns out that for each i £ {1,2}, there exists a canonical 
W^-equivariant bijection 7Tj : $j( < ^) $j(if') which maps simple roots 
to simple roots. Since each <l>j(if') is free, there is no ambiguity of the 
coefficient of a simple root in any given root of $(if'), and this will 
provide a canonical expression of roots in $j(if ) in terms of IT via 7Tj. 
However, since if lacks the integrality condition assumed in [21J, the 
proof used in [2TJ will not apply here. Further, since it is no longer 
true that there exists a bijection between $j(if) + [i = 1,2) and the 
reflections in W, the proofs used in [11] and [16] will not apply either. 

Let V[ be a vector space over E with basis nY = { a' s \ s £ S } in 
bijective correspondence with S, and let V 2 ' be a vector space over K 
with basis n' 2 = { (3' s \ s £ S }, also in bijective correspondence with S. 
Define linear maps tt\ : V{ — > V\ and 7T2 : V£ — >■ V2 by requiring that 

tti A s o/J = A s a s , and vr 2 (J^ M s /^) = 

for all X s ,fi s £ M, and define a bilinear map ( , )': V/ x V 2 ' -)• 1 by 
requiring that (a' s , (3' t )' = (a s , (3 t ) for all s, t £ S\ Observe that then 
( x \ y y = (7ri(ar , ),7r 2 (y / )) for all x' £ 1// and £ 

Thus if' = (S, V[, V2, n 2 , ( , )') is a free Coxeter datum with 
the same parameters as if, and therefore is associated to the same 
abstract Coxeter system (W,R). Applying Theorems 12.101 to if' then 
yields isomorphisms f[ : W -> V^'(if') and / 2 : W -> W 7 ^')- Th ese 
isomorphisms induce W^-actions on V[ and V^' via: vox' = (f[(w))(x'), 
and uy' = Cf 2 ( M ))(j/) for a11 w,u eW, x' e V{ and y' £ V 2 . Note that 
for each % £ {1, 2} and each s £ 5, it follows from the above definitions 
that /j'(r s ) = pv(s), and this in turn yields that 7Tifl(r s ) = fi(r s )ni, 
where fi and f'2 are as in Theorem 12.101 Since W is generated by 
{ r s I s £ S}, it follows that iiif-(w) = fi(w)iTi for all u> £ W and 
z £ {1,2}. Summing up, one has that 

iTi(wz') = WTTi(z') for all w £ W and z £ V^, (3.1) 

and hence each 7Tj is a IV-module homomorphism. 

Proposition 3.5. For each i £ {1,2} ; the restriction of 7Tj defines a 
W-equivariant bijection $j(^') — >■ $j( < ^'). 

To prove Proposition [331 we need a few elementary results and some 
further notations. 
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Definition 3.6. For each i G {1,2}, define an equivalence relation 
~i on as follows: if Z\ and z 2 G then Z\ ~j z 2 if and 

only if Z\ and z 2 are (nonzero) scalar multiples of each other. For 
each z G write "z for the equivalence class containing z, and set 

^0 = {z\ze<s> i (tf)}. 

Observe that the action of W on (for % = 1, 2) gives rise to a 

well-defined action of II 7 on satisfying = wtz for all w G VT, 

and all z G $i(^). 

Definition 3.7. For i G {1, 2}, and for each w G W, define 
Ni(w) = {7 I 7 e $+(^) and w 7 G $r(tf) }. 

Note that for w G W, the set Ni(w) (i = 1,2) can be alternatively 
characterized as {7 | 7 G <S>~ i^?) and u>7 G <&f{ftf) }. Hence 2" G Ni(w) 
if and only if precisely one element of the set {z,wz} is in $^(^). A 
mild generalization of the techniques used in ((TTJ 5.6 Proposition]) 
then yields the following result: 

Lemma 3.8. (i) If s e S then Aq(r s ) = {a s } and N 2 (r s ) = {j3 s }. 

(ii) Let w & W. Then Ni(w) and N 2 (w) both have cardinality i(w). 

(iii) Let W\, w 2 G W and let -j- denote set symmetric difference. Then 
Ni(wiw 2 ) = w^Niiwi) + Ni(w 2 ) for each i G {1, 2}. 

(iv) Let W\, w 2 G W. T/ien /or eac/i i G {1, 2}, 

£(wiW2) = £(wi) + £(^2) i/ o^<^ only if Ni(w 2 ) C Ni{w\w 2 ). 

□ 

If Wi,w 2 G iy with ^(w^w^) = ^(it>i) + ^(^2) then we call w 2 a 
n<?At /iand segment of w 1 mj 2 - If w = r Sl r S2 • - • r si (where w G W and 
si, • • • , si G S 1 ) with £(u>) = /, then the above lemma yields that 

Ni{w) = {a; t , r si a^, r S; r S; _ 1 5^, . . . , r si r si _ x . . . r S2 a^} (3.2) 

and 

N 2 (w) = {(3 Sl , r S! /Vi, Ts^s^Ps^ r Si r S; _ 1 . . .r S2 /3 Sl }. (3.3) 

Essentially the same argument as in [TSJ Lemma, Lecture 3] yields 
the following: 

Lemma 3.9. W is finite if and only if is finite ( for i = 1,2). 

□ 

Remark 3.10. Let K C S 1 . If we define Vij^ to be the subspace of Vi 
spanned by Hik = { a s \ s G K} and V 2 k to be the subspace of V 2 
spanned by H 2 k = { (3 S I s G K }, and let ( , )k be the restriction of 
( , ) to Vik x V 2K , then clearly = (K, V 1K , V 2K , Hik, LW, ( , )x) 
is a Coxeter datum with parameters {m st \ s,t £ K }. Next we write 
Wk = ({ r£ I s G if }) for the corresponding abstract Coxeter group, 
and let 77 : W^- — > W be the homomorphism defined by i-» r s for all 
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s G K. It follows immediately from the formulas for the actions of W 
on V\ and Wk on V\k that r' s v = r s v for all s G K and v G V\k-, and 
therefore = T](w)v for all G Wk and t> G Vi^. Since the action of 
Wk on V\k is faithful, it follows that 77 is injective. Thus Wk can be 
identified with the standard parabolic subgroup of W generated by the 
set { r s | s G K }. 

Definition 3.11. Given AT C 5, define $ik(C V 1K ) and $ 2 ^(C \Z 2 x) 
to be the root systems for Wk corresponding to ^k, and <&i K , &2K t° 
be the corresponding sets of positive roots. 

In other words, we have = {wa r \ w G Wk and r G K}, and 
= $ix n PLC(IIik); similarly for & 2 k and 

Remark 3.12. It is a particular case of Lemma [3.91 that Wk is finite 
if and only if either or $ 2 x is finite. 

Remark 3.13. It follows easily from Lemma 13.21 (ii) that if Wk is 
finite then there exists a (unique) longest element % G Wk satisfying 
the condition Ni(wk) = $ik } — • F° r present purposes 

we require this only in the special case when K has cardinality 2. 

Notation 3.14. Forr, s G S, if m rs < oo then ({r r ,r s }) is finite, and 
its longest element, denoted by W{ rtS y, is r r r s r r ■ ■ ■ = r s r r r s ■ ■ ■ , where 
there are m rs alternating factors on each side. 

Lemma 3.15. Suppose that s,t G S and w G W such that wa s = Xa t 
for some positive constant X. Let r G S be such that £(wr r ) < £{w). 
Then 

(i) ({r r , r s }) is finite. 

(ii) iV\(u; {r . s} r s ) = $i{ r , s } \ {d s }. 

(iii) £(wr s w^ s} ) = £(w) - £(w{ r>s yr s ). 

Proof, (i) Since £(wr r ) < £(w), it follows from Lemma 13.21 (ii) that 
wa r G Q^i^o). Furthermore, wa^ ^ c7 t , for otherwise = c7 s , contra- 
dicting Lemma [2.51 Observe that then 

wr s a r = w(a r — 2(a r , (3 s )a s ) = wa^ — 2X(a r ,f3 s )a t 

S &7(V) \ ]R{at} a scalar multiple of at 

Thus it can be checked that wr s a r G $5~(^). Now the fact that both 
wr s a r and wr s a s are negative implies that wr s (X'a r +fi'a s ) is a negative 
linear combination of IT whenever A', ^ > 0. This says precisely that 
&i{r,s} ^ Ni(wr s ). Since Ni(wr s ) is a finite set of size £{wr s ) by 
Lemma T3.8I (ii). it follows from Remark 13. 121 above that ({?•>, r s }) must 
be finite. 

(ii) First, W{ riS y exists by part (i). Next let fia r + va s G Qfiff) 
(where /i, v > 0) be arbitrary. Then it follows from Remark 13.131 that 
W{ r ^r s (na r + ua s ) G Qfiff) if and only if r s (/i« r + z/a s ) G $^(^), 
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which by Lemma I3TB1 (i) happens if and only if \x = 0, and consequently 

Ni(w{ r>8 yr a ) = ®i{r,s} \ {a s }i as required. 

(iii) By Lemma [3.81 (iv), to show that wu s yr s is a right hand seg- 
ment of w, it is enough to show that Ni(w{ r>s yr s ) C Ni(w). Sup- 
pose that a G such that a G Ni(w{ rtS yr s ). By (ii) above 
we may write a = C\Ot T + C2<y s for some C\ > and c 2 > 0. Then 

= C\wa r + c 2 A«i. Suppose for a contradiction that toa G 
Then c 2 Aa t = wa — C\wa r G PLC(n 1 ). Rearranging this equation 
gives X'a t = J2 t / eS \{t} ^t' a t' > wnere A' is a constant and A# > for 
all t' E S \ {£}. Now if A' > then we have a contradiction to 
Lemma 12.51 on the other hand if A' < then we have a contradic- 
tion to i PLC (Iii). □ 



Now we are ready to prove Proposition 13.51 

Proof of Proposition \37b\ Since = { wa' s \ w G W and s G S }, 

to prove that the restriction of 7Ti to $i(^") is bijective it suffices to 
show that if iri(wa? s ) = iti{ya>' t ) for some w, v G W and s, t G S, then 
TOj = va' t . Observe that TCi(wa' s ) = wa s , and iti(vo/ t ) = va t . Hence it 
suffices to prove the following statement: if wa s = Xa t for some w G W, 
s, t G S, and A / 0, then wa' s = \a' t . We assume that wa s = Xa t , 
and proceed by an induction on £(w). The case £(w) = reduces to 
the statement: if a s = \a t for some s, t G S then a! s = \a' t . Given 
a s = Xat, Lemma [2.51 and the requirement that ^ PLC(Ili) together 
yield that A = 1 and s = t, and we are done. Thus we may assume 
that £(w) > 0, and choose r G S such that £(wr r ) < £{w). Lemma [3. 151 
yields that ({r r , r s }) is a finite dihedral group (hence m TS is finite), and 
£{w{w{ riS }r s )~ l ) = £{w) —£{w{ r ^r s ). We treat separately the cases m rs 
even and m rs odd. 

If m rs = 2k is even, then Ws rtS \ = (r r r s ) k = (r s r r ) k , and then the 
formulas in Lemma 12.61 (i) yield 

{w{ r , s }r s )a s = -w {r , s} a s = -{r s r r ) k a s 

sin((m rs + l)n/m rs ) — cos(n / m rs ) sin(7r) 

-Cts ; 7Ti : — ; — ; r«r 



sin(7r/m rs ) " (a r ,[3 s ) sm(ir/m rs 

= a s ; (3.4) 

and by exactly the same calculation in V[ we have: 

{w {rtS} r s )a' s = a' s . (3.5) 

Observe that f!3.4p yields that 

a t = wa s = w(w[ rtS} r s )~ 1 (wi r)S yr s )a s = w(w{ r!S yr s )~ l a s . (3.6) 

Now since £(w(w{ riS }r s ) _1 ) < £(w), the inductive hypothesis combined 
with (13. 6 p give us 

a' t = w(w {r:S} r s y l a' s . (3.7) 
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Then it follows from (13. 5j) and ( 13. 7p that 

wa' s = (wiw^syr^iwfrsyrs))^ = w(to {r)S} r s )~V s = a' t , 

and the desired result follows by induction. 

Next if m rs = 2k + 1 is odd, then W{ r ^r s = ; . . r r r s r r ^ = {r s r r ) k . 

(m rs — 1) factors 

Then the formulas in Lemma 12.61 (i) yield that 

{w{r,s}r s )ot s = (r s r r ) k a s 

sin(7r) — cos(7r/m rs ) sin(2/c7r/m rs ) 
~~ « s H ; — — — ; —ot r 



sin(7r/m rs ) (a r ,(3 s ) sm(n/m rs ) 

-cos(7r/m rs ) 

a r ; (3. 



and by exactly the same calculation in V[ we have: 

/ \ / -cos(vr/m rs ) , 

(w {r , s} r s )a s = — a r . (3.9) 

Observe that ( 13.81) yields that 

/ \ / n-1 , — cos(7r/m rs ) 
a t = w(w {r:S} r s ) {w{ r>s} r s )a s = w{w {r!s} r s ) ( — a r ). 

\Oiri Ps) 

(3.10) 

Since £(w(w{ rtS yr s ) x ) < £(w), combining (13. lOj) and the inductive hy- 
pothesis yield that 

/ w -cos(7r/m rs ) 
a t =w{w {riS} r s ) ( — a r ). (3.11) 

Finally it follows from fl32J and fl3TTT|) that 

/ / \ / / \-i / — cos(7r/m rs ) t 

wa s = w[w {r>s} r s ) {w {r>s} r s )a s = w(w{ r>s} r s ) ( — — — a r ) 

\pi r , p s ) 

= ®'t, 

completing the proof that 7i"i restricts to a bijection $i(^") <H- $i(^). 
Exactly the same reasoning also yields that iT2 restricts to a bijection 
$ 2 (^') O $ 2 (^). □ 

Under the freeness condition of each root a' G $i(^") can be 
written uniquely in the form a' = ^ sgS A s a' s ; we say that A s is the 
coefficient of a' s in a', and denote it by coeff s (a'). Similarly each root 
0' G $2(^") can be written uniquely as (3' = J2ses ^sP' 8 \ we say that 
fi s is the coefficient of f3' s in (3', and denote it by coeff s (/?'). 

Definition 3.16. (i) Let a G $i(^) be arbitrary. For each s E S, de- 
fine the canonical coefficient of a s in the root a, written coeff s (a), by 
requiring that coeff s (a) = coeff s (7rj 1 " 1 (a)), where tti is as in Propo- 
sition 13.51 The support of the root a, written supp(a), is the set 
{ a s I coeff s (a) 7^ }. 
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(ii) Let (3 G $2(^0 be arbitrary. For each s G S, define the canonical 
coefficient of {3 S in the root (3, written coeff s (/3), by requiring that 
coef£ s ((3) = coeff s (7r 2 ~ 1 (/3)), where 7r 2 is as in Proposition 13.51 The 
support of the root 0, written supp(/3), is the set { (3 S | coeff s (/3) 7^ }. 

A similar proof as that of Proposition 13.51 establishes the following: 

Proposition 3.17. Suppose that s,t G S. If wa s = Xat for some 
w G W and some non-zero constant X, then wf3 s = j-flt. □ 

In particular, Proposition 13. 171 implies the following: if Wia s = ui2Ct t 
for some s,t G S and 1^1,^2 G W, then Wi(3 s = w 2 (3 t . Thus there 
exists a well-defined map $i(^) —> $2(^0 satisfying the requirement 
that wa s wf3 s for all s G S and w G W. This is clearly the unique 
W-equivariant map $i(^) —> $2(^) satisfying a s H- (3 S for all s G S. 
Furthermore, by going through a similar reasoning as in the proof of 
Proposition 13. 5[ we may conclude that this map is a bijection. 

Definition 3.18. Let : $i(V) ->■ $ 2 (X) be the W^-equivariant bijec- 
tion satisfying 0(a s ) = (3 S for all s G S. 

Remark 13.31 then immediately implies the following: 

Corollary 3.19. Let a G Then a G §f(jt?) if and only if 

0(a) G &2 (X); and similarly a G $i~C^) anc? or % 2/ 0(a) e $2 0^)- 

□ 

Note that Proposition 13.171 can be generalized to the following: 

Lemma 3.20. Suppose that a G and suppose that X is a non- 

zero constant such that Xa G $i(^). T/ien 0(Aa) = ^0(a). 

Proof. Write a = wa s for some w E W and s G 5. The fact that 
Xa G implies that u; _1 (Aa) = Aw _1 a = Aa s G Then 

0(Aa s ) = j(fi(a s ) by Proposition 13.171 Now it follows from the W- 
equivariance of that 

0(Aa) = (p(Xwa s ) = 4>(wXa s ) = w(p(Xa s ) = —w(f>(a s ) = —0(a). 

A A 

□ 

Definition 3.21. Let & = ( S, V x , V 2 , Tlx, n 2 , ( , ) ) be a Coxeter datum. 

(i) For each i G {1,2} and each z G Qfffi), define the depth of z 
(written dp^j(z) ) to be 

dp^ ^z) = min{£(w) | w G W and wz G }■ 

(ii) For each i G {1,2} and Zi,z 2 G $^(^), write Z\ <i z 2 if there 
exists w & W such that 22 = wzi, and dp^,^) = dp^j(zi) + £(w). 
Furthermore, we write Z\ <i z 2 if Z\ <i z 2 but z\ 7^ z 2 . 

A mild generalization of Lemma 1.7 of [3] yields the following: 
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Lemma 3.22. Suppose that = ( S, Vi, V 2 , III, II 2 , ( , ) ) is a Coxeter 
datum. Letse S, ae $J"(if) and /9 G $2W T/ien 

dp»,i(a) if(a,p 8 ) = 0, 
<Wi( a ) + 1 tffaPe) < 0; 



and 



dp^, 2 (/?) - 1 if(<*s,P) > 0, 
dp^O*^) = { <W i2 (/3) »/ (a fl , P) = 0, 
dp^ 2 (/?) + l i/<a„/3) <0. 



□ 



Lemma 3.23. Suppose that & = ( S, Vi, V 2 , III, II 2 , ( , ) ) is a Coxeter 
datum, and a G $^( < ^ 7 ). JTien dp<^ 1 (a) = dp^ 2 (0(a)). 

Proof. Let u> G W be such that wa G and d P^,i( a ) = £{w). 

Then Corollary 13.191 yields that (f)(wa) G $ 2 (^), and it follows from 
the W-equivariance of <f> that, w(<p(a)) G $2 Hence 

dp^, 2 (0(a)) < £(w) = dp^a). 
By symmetry, dp^ x (a) < dp^ 2 (0(a)), whence equality. □ 

Lemma 3.24. Suppose that = ( S, Vi, V 2 , LTi, Il 2 , ( , ) ) is a Coxeter 
datum. Let a G $i(^) and s G S. Then 

(a, /3 S ) > if and only if (a s , </>(«)) > 0; 

and 

(a, P s ) = if and only if (a s , 0(a)) = 0; 

and 

(a, p s ) < if and only if (a s , 4>{a)) < 0. 

Proof. Combine Lemma 13.221 and Lemma 13.231 then the desired result 
follows. □ 

In fact, the above gives rise to a more general result: 

Corollary 3.25. Suppose that c € = ( S, Vi, V 2 , IL, U 2 , ( , ) ) is a Cox- 
eter datum, and ai,a 2 G $i(^). Then 

(«i,0(a 2 ))>O if and only if (a 2 , > 0; 

and 

(ax, 0(a 2 )) = and only if (a 2 , = 0; 

and 

(«i,0(a 2 ))<O if and only if (a 2 , < 0. 

Proof. Write a 2 = wa s for some w G IV and s G S 1 . Given the W- 
invariance of ( , ) and the W-equivariance of 0, we have 

(ai,0(a 2 )) = (oti,wp s ) = {w^a^Ps). 
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It then follows from Lemma [3.241 that (w 1 ai,(3 s ) > precisely when 

(a s ,0O -1 ai)) = (wa 8 ,<f>(ai)) = (a 2 ,4>(a 1 )) > 0. 
The rest of the desired result follows in a similar way. □ 

4. Comparison with the Standard Geometric Realization 

of Coxeter Groups 

In this section we recover the root systems of Coxeter groups in 
the sense of [13] or [17] as special cases of root systems arising from a 
Coxeter datum. Subsequently we give comparison results between such 
special cases and the more general root systems arising from a Coxeter 
datum. These comparisons will provide useful reduction of the non- 
orthogonal representations studied in the previous sections into those 
of [E] or H7J. 

Fix a Coxeter datum ^ = ( S, Vi, V2, Hi, II 2 , ( , ) ), and let V be 
a vector space over R with a basis Il = {7 s |sG5'}in bijective 
correspondence with 5*. Suppose that ( , ) : V x V — > K is a symmetric 
bilinear form satisfying the following conditions: 

(CI') ( 7s , 7 ,) = Moralised; 

(C2') (7„7 t ) < 0, for all distinct s,t e S; 

(C3') ( 7s , 7i ) 2 = (a s ,p t )(a t ,p s ), for all s,t e S. 

Then c €" = ( S, V, V, IT, IT, ( , ) ) is a free Coxeter datum with the same 
Coxeter parameters m st , (s,t e S) as those of and hence ^ and 
c &" are associated to the same abstract Coxeter system (W,R). It is 
easily checked that = ^ 2 ( c ^'"), allowing us to write $ in 

places of $i( < ^ // ) and $2(^")- Furthermore, we write <3> + and $~ for 
the corresponding set of positive roots and negative roots respectively. 
It is also readily checked that W can be faithfully embedded into the 
orthogonal group of the bilinear form ( , ) on V. This recovers the 
orthogonal representation of W as defined in [2], [13] and [17] . We refer 
to such V as the standard geometric realization (Tits representation) 
of W or simply the standard Tits representation of W. 

It follows from Lemma 12.131 that ( , ) is VF-invariant. It is well- 
known (and can be readily checked) that if x and Xx are both in $ 
for some constant A, then A = ±1. Since c €" is free, it follows that 
each 7 e $ can be written uniquely in the form 7 = Ylses ^sls- We 
say that A s is the coefficient of 7 S in 7, and denote it by coeff s (7). 
Finally, to simplify notation, for any 7 € $ we write dp(7) in place 
of dpc (f /; 1 (7)(= dp<^;/ )2 (7)), and for 71,72 G $ we simply write 71 ^ 72 
when 71 ^ 72 (and 71 < 2 72)- 

Similar arguments as for Proposition 13.51 gives us the following: 

Proposition 4.1. There are W-equivariant maps <f>\ : $i( < ^') — > $, and 
02 : $2(^0 -> satisfying <fii(a s ) = 7^ = faiPs), for all s e S. □ 
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Remark 4.2. We stress that unlike 7Ti, 7r 2 of Proposition 13.51 and of 
Definition 13.181 the new maps 0i and 02 axe not injective in general, 
and we shall see more on this fact in Lemma 14.81 (ii) below. 

Lemma 4.3. Suppose that a G $i(^). Then a G ^( e tf) implies that 
0i(a) G $ + , and a G $r<X) implies that 0i(a) G 

Proof. We may write a = wa r , for suitable w (z W and r G S 1 . If 
a G (^), then Lemma 13.21 (ii) yields that £{wr r ) = £{w) + 1, in 
which case Lemma 13.21 (ii) applied to the Coxeter datum c tf" yields 
that 0i(a) = 4>i(wa r ) = u>0i(a r ) = wy r G $ + . Likewise we see that 
a G $r(^) implies that 0i(a) G $~. □ 

Using the same argument as in Lemma 13.231 we have: 

Lemma 4.4. Suppose that a G $^(^). Tnen dp^ 1 (a) = dp(0i(a)). 

□ 

Corollary 4.5. Suppose that a G $i(^) ; and s G S*. Then 
(0i (a), 7 S ) > i/and only if (a, f3 s ) > 0, 

and 

(0i (a), 7 S ) = and on/y if (a, /3 S ) = 0, 

and 

(0i (a), 7s) < if and only if (a, (3 S ) < 0. 

Proof. Follows from Lemma 14.41 and Lemma 13.221 applied to c £" . □ 

Next we have a well-known result on Coxeter groups (a proof can be 
found in [5], in the discussion immediately before Lemma 2.1) which 
we shall use repeatedly in later calculations. 

Lemma 4.6. Suppose that ICS* and w G W. Let Wi denote the 
standard parabolic subgroup in W corresponding to I, as defined in 
Remark VS.liA Choose w' G wWj to be of minimal length in the left 
coset of Wi in W containing w. Then £(w'v) = £(w') + £(v) for all 
v G W T . □ 

Proposition 4.7. For each a G $i(^) and each r G S , 
coeff r (a) coeff r (0(a)) > (coeff r (0i(a))) 2 . 

Proof. Replace a by —a if needed, we may assume that a G $^"( < ^'), 
and we may write a = wa s , where w G W and s G S. The proof 
is based on an induction on £(w). If £(w) = then the result clearly 
holds with an equality. Thus we may assume that £{w) > 1, and 
choose t G S such that £{wr t ) = £{w) — 1. Observe that w = u>iu>2, 
where W\ is of minimal length in the coset w({r s ,r t }), and W2 is an 
alternating product of r s and r t , ending in r t . Then Lemma [4.61 yields 
that £(w) = £(w 1 )+£(w 2 ), £(u>ir s ) = e(wi) + l, and £(wm) = £(wi) + l. 
Consequently Wia s G $i~( < ^ 7 ) and Wia t G $i~(^) by Lemma [3T21 (ii). 
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The formulas in Lemma 12.61 yield that w 2 a s = pa s + Xqa t , where A is 
a positive constant and pq > 0. If p and q are both negative, then 

a = wa s = W\W20i s = Wi(pa s + Xqa t ) = pwia s + \qwia t G $]~( < ^ 7 ), 

contradicting the assumption that a G $^(^). Therefore p, q > 0. By 
Lemma [2751 {a s ,a t } is linearly independent, hence coeff s (w 2 a s ) = p, 
and coe& t {w2<y s ) — Xq. Similar calculations as in Lemma |2~7o1 yield that 

^27s = Pis + qjt, and w 2 (3 s = p(3 s + {f5 t . 

And Lemma 12751 implies that coeff s (u> 2 /3s) = P and coeff t (w 2 /3 s ) = |. 
Next we set 

x = coeff r (a) , x' = coeff r (0(o;)) , x" = coeff r (0i(o;)); 

and 

y = coeff r (wia s ) , y = coeff r (wi/3 s ) , y" = coeff r (wi7 s ); 

and 

z = coeff r (wiQ;i) , z' = coeS r (wi(3 t ) , z" = coeff r (u>i7 4 ). 

Then 

xx' — x" 2 = (py + \qz)(py' + \qz') — (py" + qz") 2 

= V\VV' - y" 2 ) + q\zz> - z" 2 ) + P q{\yz' + Xzy' - 2y"z"). 

(4.1) 

Since £(wi) < i(w), it follows from the inductive hypothesis that 

yy > (y") 2 and zz' > (z") 2 , 

and so the first two summands in the last line of (14. ip are nonnegative. 
Next apply the geometric mean and arithmetic mean inequality to the 
terms \yz' and Xy'z , and we can conclude that jyz'+Xy'z > 2^/yy'zz' . 
But the inductive hypothesis yields that yy' zz' > (y"z") 2 , showing that 
the third summand in the last line of ()4.ip is also nonnegative, whence 
xx' — x" 2 > 0, and the desired result follows by induction. □ 

Lemma 4.8. Suppose that a G $i(^'). 

(i) Let t G S . Then 

coeff t (a) = if and only if coeff t (0 1 (a)) = 0, 
< coeffi(a) > if and only if coeff t (0 1 (a)) > 0, 
coeff t (a) < if and only if coeff 4 (0i(a)) < 0. 

(ii) The situation 4>i(a) = ±7 S (s G S) arises if and only if a = Xa S! 
for some nonzero constant X. 

Proof, (i) Lemma [372] (i) applied to the Coxeter datum c €" yields that 
$ = $+ |±| $~. Hence we only need to verify that coefft(a) = if 
and only if coefft(0i(a)) = 0, for the rest of (i) follows readily from 
Lemma fl~73l By Proposition 14. 71 we only need to show that the condition 
coefLf(0i(a)) = implies that coeff t (a) = 0. Replacing a by —a if 
needs be, we may assume that a G and write a = wa r for 



18 



FU, XIANG 



some w G W and r G S. If £(w) = then there is nothing to prove. 
Thus we may assume that £(w) > 1 and proceed by an induction on 
£(w). Choose s G S such that £(wr s ) = £{w) — 1. Now w = W\W 2l 
where W\ G w{{r r ,r s }) is of minimal length, and w 2 is an alternating 
product of r r and r s ending in r s . Then Lemma 14.61 implies that w 2 
is a right segment of w. Since a = wa r is positive, it follows from 
Lemma [3.81 (iv) that w 2 a r is positive too. Direct calculations as those 
in Lemma [2.61 show that 

W2lr = Vlr + 97s> w 2 a r = pa r + \qa s and w 2 (3 r = p(3 r + j(3 s 

for some non-negative constants p and q, and some positive constant 
A. Now Lemma 12751 yields that coeff r (u> 2 ar) = P, coeff s (w2a r ) = Xq, 
coeff r .(w 2 /3r) —P and coeS s (w 2 f3 r ) = |. Then 

= coeff 4 (</>i(ct)) = coeff((w7 r ) = coeff t (wi(p7 r + qj s )) 
= pcoeff t (wi7 r ) + qcoe&tlwijs). 

By Lemma 14761 and Lemma [372] (ii) (applied to the Coxeter datum ^"), 
Wi'jr and Wi'jg are both positive, and hence it follows from the above 
that 

pcoeff t (u>i7 r ) = gcoeffi(^i7 s ) = 0. 
Then the inductive hypothesis yields that 

p coe& t (wia r ) = qcoe& t (wia s ) = 0, 

and therefore 

coefLj(a) = coeS t (wa r ) = pcoe&t(wia r ) + Xq coeff f (wia s ) = 
as required, and (i) follows by induction. 

(ii) Follows readily from part (i) above. □ 

Remark 4.9. Suppose that (3 G $2(^)5 and t G S. Then the same 
arguments as those used in the proof of the above Lemma yield that 

coeff t (/3) = if and only if coeff t (0 2 (/3)) = 0, 
coeff t (/3) > if and only if coeff t (0 2 (/3)) > 0, 
coefft(/3) < if and only if coeff t (0 2 (/3)) < 0. 

The next result is taken from [1]: 

Lemma 4.10. (Brink [H Proposition 2.1]) Suppose that 7 G $ and 
r G S. Then coeff r (7) > implies that coeff r (7) > 1. 

Further, suppose that < coeff r (7) < 2. Then either coeff r (7) = 1 

or coeff r (7) = 2cos(^ iI — ) ; where r 1; r 2 G S with 4 < m rir2 < 00. □ 

Combining the results in Proposition 14.71 to Lemma 14.101 (inclusive) 
and an argument similar to the one used in the proof of Proposition 14. 7\ 
we may deduce the following: 
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Proposition 4.11. Let a G $i(^) and t G S be arbitrary. Then 

coeff t (a) > if and only if coeff t (0(a)) > 0, 
and in this case coeff t (a) coeS t (<p(a)) > 1. In particular, 

0(supp(a)) = supp(0(a)). 
Furthermore, suppose that 1 < coeS t (a) coeff t (0(a)) < 4. Then either 

coeff t (a) coenV0(a)) = 1, or else coefLYa) coeff t (0(a)) = 4cos 2 ( — ), 

m 

where m = m rir2 , for some r 1; r 2 G S with 4 < m < oo. In par- 
ticular, if a G ^( c tf) then coeffi(a) coeffi(0(a)) = 1 if and only if 
coefft(0i(a)) = 1. □ 

Proposition 4.12. Suppose that «i,a 2 G $i(^). Then 
(a 1 ,(p(a 2 ))(a 2 ,4>(a 1 )) > (0i(«i), 0i(a 2 )) 2 . 

Proof. Since both ( , ) and ( , ) are VT-invariant, and 0, 0i are W- 
equivariant, we may replace «i and a 2 by ua\ and «a 2 for a suitable 
u E W so that a 2 = « s for some s 6 5. Furthermore, replace oi\ by 
— a.\ if needs be, we may assume that a\ G $^"(^). We proceed with 
an induction on the depth of a±. 

If dp<^ 1 (o;i) = 1, then a.\ = Xa r , for some positive constant A and 
some r G S. It follows from Lemma I4~S1 (ii) that </>i(o;i) = 7 r . Further- 
more, Lemma [3.201 yields that <p{ct\) = jfi r , and hence 

(ai,0(a 2 ))(a 2 ,0(ai)) = X(a r , fi s )\(a s , fi r ) 

= (a r ,/3 s )(a s ,/3 r ) = (jr,^) 2 = Oi(cti), <Ma 2 )). 

Thus we may assume that dp %?1 (a 1 ) > 1. First, if (cti, fi s ) > then 
Lemma [3.221 yields that r s ai -<i cti, and hence 

= (r s a 1 ,r s fi s )(r s a s ,r s (f)(a 1 )) 

= (-(r s ai,/3 s ))(-(a s ,<Kr s ai))) 

> (0i(r s «i), 7 S ) 2 (by the inductive hypothesis) 

= Oi(ai),-7 s ) 2 

= (0i(ai),7,) 2 , 

as required. Thus we may further assume that {ai, fi s ) < 0. 

Next let t G S be such that r t oi\ -<\ a.\. Then Lemma T3.22I yields 
that («!, fi t ) > 0, and, in particular, s ^ t. Now let w £ W be 
a maximal length alternating product of r s and r t ending in r t such 
that dp vi (wai) = dp vi (ai) — £(w). In particular, w ^ 1. Thus 
dp^ < dp C(f !(a;i), and so the inductive hypothesis yields that 

(wa 1 ,fi s )(a s ,(f)(wa 1 )) > (j s , ^(wa^) 2 (4.2) 

and 

(wai,fi t )(a u (t){wa 1 )) > (j t , 0i(u>cti)) 2 . (4.3) 
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Next, for any reduced expression w = r si - ■ -r Sl (sj e S), it is readily 
checked that r Sl ai -<x a\, and so it follows from Lemma 13.221 that 
{ai, (3 Sl ) > 0. Therefore si ^ s, and w has no reduced expression 
ending in r s . That is, w is a product of r s and r t with strictly fewer 
than m s t factors. Thus by Lemma f2. 11 l or a direct calculation as that in 
Lemma [276| there are non- negative constants p, q and positive constant 
A such that 

w ls = Vis + lit and wa s = pa s + Xqa t and w/3 s = pf3 s + j(3 t . 
Thus 

{a!, (3 s )(a s ,4>(a 1 )) - (7 s ,0i(«i)) 2 
= (wai,w/3 s )(w;a s ,0(wai)) - (wj s , <fii(wai)) 2 

( since ( , ) and ( , ) are PF-invariant) 
= (wa lt pP a + ip t )(pa a + \qa t , ^(waj) - {$i(wa 1 ),pr{ s + q^ t f 
= p 2 ((wa 1 ,f3 s )(a s ,(j)(wai)) - (^i(iwai), 7 S ) 2 ) 

V v ' 

A 

+ g 2 ((wai,/3 t )(a 4 ,0(wai)) - (0i(wai),7 4 ) 2 ) + C. 

V v ' 

13 

where 

C = pq{ j(waii, Pt) (a s , 4>{wai)) + X(wai, f3 s ) (a t , <p{wai)) 
- 2(0i(wai),7 s )(0i(wai),7t) ). 

It follows from (14. 2 p and ( 14. 3 p that A and B are both nonnegative. It 
follows from the geometric mean and arithmetic mean inequality that 

\(wa u p t )(a s ,(j){wa 1 )) + \{wa x , /3 s )(a t , <j>(wan)) 

> 2a/ (toi, (3 s ){a s , <j)(wa 1 )){wai, (3t){a t , <p{wai)) 

> 2( ( f> 1 (wa 1 ),j s )(<p 1 (wa 1 ), lt ) (by (Q and flM}), 

that is, C > as well. Therefore (cci, {3 s )(a s , 4>{a.i)) > (j s , 0i(c*i)) 2 , 
and the desired result follows by induction. □ 

5. Tits cones and a non-positivity result 

Let = ( S, Vi, V2, III, n 2 , ( , ) ) be a fixed Coxeter datum, and let 
(W, R) be the corresponding Coxeter system. In this section we study 
a class of cones associated to ^ that are analogous to the Tits cones 
in the classical setting (as defined in |22j or [17J). Furthermore, we 
investigate certain ^-invariant sets in V\ and V% that are closely related 
to these cones. The key result of this section is a generalization to 
Proposition 1.2 of [20] and Proposition 3.4 of [IB] . For this section we 
impose one additional condition on ff, namely: 

(C6) Vi = span(II 1 ) and V 2 = span(n 2 ), 

and we retain all other conventions and notations of earlier sections. 
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Notation 5.1. For each i G {1,2} and I C S, recall the notations 
Uu and Vu introduced in Remark \3.1(\ and set Pu = PLC(ILj-) U {0}. 
When I = S we write Pi in place of Pis- 

For each i G {1,2} and subset / of S recall the notation Wj (in- 
troduced in Remark 13.101) of the standard parabolic subgroup of W 
corresponding to /, it is clear that Wi acts (faithfully) on Vu. This al- 
lows us to specify a Wj-action on RomiVu, R) as follows: if w G Wi and 
g G Hom(X^/,R) then wg G Hom(14r,R) is given by (wg)v = g(w~ 1 v) 
for all v G Vu. Naturally, when I = S the Coxeter group W acts on 
Hom(Vi,]R) in a similar way. 

Notation 5.2. For each i G {1,2} and subset I of S we set 
P% = {fe Hom(y iJ , R) | f(x) > for all x G P u }, 

and 

P ## = { xe ViI | f{x) > for all f G P*}. 

Moreover, we set Uu = [j weWl wPjj > an( ^ write 

U* = {xe V tI I f(x) > for all f G U a }. 

When I = S we write P* , P** , C7» and U* in places of P* s , P* s * , U iS 
and Ufg respectively. 

We call a (convex) subset of a real vector space a cone if it is closed 
under addition and multiplication by positive scalars. It is clear that 
Pu, Pu an d Pu* (i = 1, 2) are cones for each ICS. 

Lemma 5.3. For each i G {1, 2} and each f G Hom(V^,lR) ; set 

Neg(/) = {x G $i | x G and /(x) < 0}. 
Then Ui = {f G Hom(V^M) | | Neg(/)| < oo}. 

Proof. It is enough to prove the C/i case. Let / G U\ be arbitrary. Then 
/ = for some w6 W and g G P*. Suppose that x G ^ such that 
/(x) < 0. Then (wg)x = g{w~ l x) < 0. Since g G Pf, it follows that 
w^x G $7, that is, x G A^i(w _1 ). Now since iVi(u7 _1 ) is of finite size, 
it follows that UjCj/G Hom(y x ,R) | |Neg(/)| < oo}. Conversely, 
suppose that / G Hom(K,M) with |Neg(/)| < oo. If |Neg(/)| = 
then / G Pf C C^. Thus we may assume that |Neg(/)| > 0, and 
proceed with an induction. Observe that if | Neg(/)| > then there 
exists some s G S such that f(a s ) < 0. It is then readily checked that 
|Neg(r a /)| = |Neg(/)| — 1, and hence it follows from our inductive 
hypothesis that r s f G U\. Since U\ is clearly ^-invariant, it follows 
that / G Ui, and therefore {/ G Hom(Vi,R) I I Neg(/)| < oo} C Ui, as 
required. □ 

The above lemma yields that U\ and U2 are cones. In fact, for each 
/ C S we can show that Uu and U21 are cones. These cones generalize 
the notion of the Tits cones as defined, for example, in [XTf 5.13]. 
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Definition 5.4. We call XJ\ and U 2 the Tits cones of the Coxeter 
datum c €; and for each / C S we call Uu and L^/ the Tits cones of the 
the Coxeter datum tfj = ( I, Vu, V 2 i, Uu, U 2I , (, )j ) (where (, )/ is the 
restriction of (, ) to Vu x V 2 i). 

For each i G {1,2} observe that: 

[/f = { v G I (wf)v > for all f e P* and w e W } 

= f]{veVi\ f{w- x v) > for all / G P* } 



P| { wv G K I /(v) > for all f e P* } 



(5.1) 



n ^ 



wen 7 
and similarly, 



U* = P| wP# # , for each ICS. (5.2) 



The following is a well-known result (a proof can be found in [TBI 
Notes (b) and (c), Lecture 1]). 

Lemma 5.5. Let I C. S be finite. For each i G {1,2}, the condition 
^ PLC(ilj/) is equivalent to the existence of an fa G Hom( Vi/, M) 
such that fi(x) > for all x G Uu- Furthermore, Pu = P%i- □ 

Lemma 5.6. For each subset I of S and for each i G {1,2} we have 

P** n v w c p# # . 

Proof. For each z G {1,2} write V, = Vu ® V- T where V- T denotes a 
(vector space) complement of Vu in V*. Clearly, every g>j G P* gives 
rise to a linear functional g\ in P^ as follows: for any v G V, there is 
a unique decomposition of the form v = vu + f[r where vu G Vij and 
f - 7 G V^j, and we simply set g'^v) = gi{vu). Now let v G P* # fl Vu, 
and / G P* be arbitrary. Then f(v) = f'(v) > (because /' G Pf 
and u G P* # ), and hence it follows that v G P* # , as required. □ 



Let Xi G Vj, (i — 1, 2) be arbitrary. Given condition (C6) of % \ it 
follows that Xi G Vu for some finite subset I of S. From this fact we 
may prove the following: 

Lemma 5.7. Suppose that X{ G Uf (for each i G {1,2}). Let I be a 
subset of S such that Xj G V^j. Then X{ G £7*. Furthermore, if I is 
finite then Xj G Pjj- 
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Proof. Observe that for each i G {1,2}, 

Xieufn Vu 
c f| (u>/f # n v u ) 

w£Wi 

= w (-P** H Vu) (since Wj preserves Vu) 

C p| W P** = U* (by Lemma EE]). 

Finally, it follows from Lemma 15.51 that Xi G f\ueWi w Pu — Pa when- 
ever I is finite. □ 

Now we are ready for the central result of this section: 

Theorem 5.8. (vi,v 2 ) < 0, for all V\ G Uf and v 2 G Uf. 

Proof. Suppose for a contradiction that there are «i 6 ™ and v 2 G 
with (vi,V2) > 0. Replace v 2 by a positive scalar multiple of itself if 
necessary, we may assume that (vi,v 2 ) = 1. Let / be a finite subset 
of S with v\ G Vij, and let J be a finite subset of S with v 2 G V 2 j. 
Next, set K = I U J. Then Lemma 15.71 yields that t>i G t^jjo and 
t>2 G U 2 k- Since X is finite, it follows from Lemma 15.51 that there are 
linear functional f\ G Hom^^M) and f 2 G Hom.(V 2 K, M) such that 
/i(a) > 1 for all a G n 1A: , and /2(/3) > 1 for all f3 G H 2 k- Now set 

=fi/ = { x G | f 2 (x) < f 2 (v 2 ) and (z, x) > 1 

for some 2 G C/^ with < fi(vi) }. 

Observe that =2/ 7^ 0, since t> 2 G 

Next, put e = pq^^y- We claim that for any given x & there 
exists y E with /2G/) < /^(aO ~ e - If we could prove this claim, then 
starting with x — v 2 , a finite repetition of this process will produce some 
t/G^C C P 2K with / 2 (y ) being negative, contradicting the fact 
that f 2 (y) G f 2 (P 2 K) Q (0, 00). Thus all that remains to do is to prove 
the above claim. Given arbitrary x G let z = ^ QgrilK A Q a; G £7^ 
be such that (z,x) > 1, f\(z) < /i(t>i), and X a > for all a G IIlr-. 
Note that these conditions imply that J2 a en 1K ^a(&, x ) > 1, which in 
turn implies that X aso (a SQ ,x) > r^r for some s G K. Then 

1 1 - e 

(a '°' x> - a^-TiKM "2' 

since A asQ < /i(^) < fi(vi). Set y = r SQ x. Observe that (15. 2p indicates 
that [/|^- is W^-invariant, and given that r so G , we have y G CT^. 
Moreover, 

Mv) = M x ) ~ 2 (^s ,x)f 2 (/3 S0 ) < f 2 (x) - e < f 2 (v 2 ). 
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Thus to establish our claim, we only need to show that y G £?, and 
this in turn amounts to finding some t G Uf K such that (t,y) > 1, and 
flit) < fi(vx). First, suppose that (z,f3 SQ ) > 0. Put t = r SQ z. Since 
Uf K is Vl^-invariant and z G Uf^, it follows that t G Uf K . Moreover, 

(t,y) = (r So z } r So x) = (z,x) > 1 

and 

/x(t) = f 1 (z)-2(z,p so )f 1 (a So ) < h{z) < f x (vi), 

thus proving y G srf in the case {z,(3 SQ ) > 0. Next, suppose that 
(z,/3 SQ ) < 0. Then t = z will do, indeed, 

(z,v) = (z,x- 2(a S0 ,x)/3 S0 ) 

= (z,x) - 2(a S0 ,x}(z,f3 S0 ) 

> (z,x) 

>1, 

and by our construction, z G Uf K and /i(-z) < fi(vi), thus proving 
?/ € ^ in the case (z,/3 So ) < too. This completes the proof of the 
claim, and hence the theorem follows. □ 
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